Surgical robot and method of controlling the same

ABSTRACT

Disclosed herein is a surgical robot including a slave device performing a surgical operation upon a patient and a master device controlling the surgical operation of the slave device. The slave device includes an image capture unit including a first lighting unit radiating visible light, a second lighting unit radiating UV light, and a camera capturing a visible-light image and a surgical tool coated with a UV reactive material emitting light in response to UV light radiated by the second lighting unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Korean Patent Application No.10-2013-0026616, filed on Mar. 13, 2013 in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference.

BACKGROUND

1. Field

Embodiments of the present disclosure relate to a surgical robot capableof efficiently separating surgical tools from a background imageacquired by a camera and a method of controlling the same.

2. Description of the Related Art

Minimally invasive surgery generally refers to surgery capable ofminimizing incision size and recovery time. Minimally invasive surgeryis different from laparotomy, which uses relatively large surgicalincisions through a part of the human body (e.g., the abdomen). Inminimally invasive surgery, after forming at least one small incisionhole (or invasive hole) of 0.5 cm to 1.5 cm through the abdominal wall,an operator inserts an endoscope and surgical tools through the incisionhole, to perform surgery while viewing images provided by the endoscope.

In comparison with laparotomy, minimally invasive surgery causes lesspost-operative pain, faster recovery of bowel movement, earlierrestoration of ability to eat, shorter hospitalization, faster return todaily life, and better cosmetic effects owing to the small incisionsize. Due to these properties, minimally invasive surgery is used forcholecystectomy, prostatic carcinoma surgery, hernia repair,hysterectomy, and the like, and applications thereof continue to grow.

In general, a surgical robot used in minimally invasive surgery includesa master device and a slave device. The master device generates acontrol signal in accordance with manipulation performed by a doctor andtransmits the control signal to the slave device. The slave devicereceives the control signal from the master device and performsmanipulation required for surgery on a patient. The master device andthe slave device may be integrated, or may be separately arranged in anoperating room.

The slave device includes at least one robot arm. A surgical tool ismounted on an end of the robot arm and inserted into a human body toperform a surgical operation at a surgical region in a patient inaccordance with a control signal received from the master device. Thus,in such minimally invasive surgery and laparoscopic surgery using asurgical robot, the position of the surgical tool inserted into thepatient needs to be accurately detected to control the operation of thesurgical tool.

SUMMARY

Therefore, it is an aspect of the present disclosure to provide asurgical robot capable of accurately estimating location information ofa surgical tool inserted into a patient's body and a method ofcontrolling the same.

Additional aspects of the disclosure will be set forth in part in thedescription which follows and, in part, will be obvious from thedescription, or may be learned by practice of the disclosure.

In accordance with one aspect of the present disclosure, a surgicalrobot includes a slave device performing a surgical operation upon apatient and a master device controlling the surgical operation of theslave device. The slave device includes an image capture unit includinga first lighting unit radiating visible light, a second lighting unitradiating UV light, and a camera capturing a visible-light image and asurgical tool coated with a UV reactive material emitting light inresponse to UV light radiated by the second lighting unit.

In accordance with another aspect of the present disclosure, a method ofcontrolling a surgical robot including a slave device performing asurgical operation upon a patient and a master device controlling thesurgical operation of the slave device includes performing a surgicaloperation by inserting a surgical tool coated with a UV reactivematerial into a patient's body, radiating visible light and UV lightinto the patient's body, acquiring a first image during visible lightradiation and a second image during UV radiation, and calculatingposition and direction of the surgical tool using the acquired firstimage and second image.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects of the disclosure will become apparent andmore readily appreciated from the following description of theembodiments, taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 is a diagram illustrating an outer appearance of a surgicalrobot;

FIG. 2 is a block diagram illustrating constituent elements of asurgical robot;

FIG. 3 is a first image acquired under visible light;

FIG. 4 is a second image acquired under UV light;

FIG. 5 is a flowchart sequentially illustrating a method of controllinga surgical robot;

FIG. 6 is a flowchart illustrating an example of operation S540 of FIG.5 in detail;

FIG. 7 is a flowchart illustrating another example of operation S540 ofFIG. 5 in detail; and

FIG. 8 is a flowchart illustrating operations performed after operationS540 of FIG. 5.

DETAILED DESCRIPTION

The aspects, particular advantages and novel features of the embodimentsof the present disclosure will become apparent with reference to thefollowing detailed description and embodiments described below in detailin conjunction with the accompanying drawings. In the drawings, the sameor similar elements are denoted by the same reference numerals eventhough they are depicted in different drawings. In the followingdescription of the embodiments, a detailed description of knownfunctions and configurations incorporated herein will be omitted when itmay make the subject matter of the embodiments rather unclear. Herein,the terms first, second, etc. are used simply to discriminate any oneelement from other elements, and the elements should not be limited bythese terms.

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an outer appearance of a surgicalrobot. FIG. 2 is a block diagram illustrating constituent elements of asurgical robot.

A surgical robot may include a slave device 200 to perform surgery on apatient P who lies on an operating table, and a master device 100 toremotely control the slave device 200 in accordance with manipulation byan operator S (e.g., a doctor). In this regard, at least one assistant Aassisting the operator S may be positioned near the patient P.

Here, assisting the operator S may refer to assisting a surgicaloperation performed in accordance with manipulation by the operator Swhile surgery is in progress, such as replacing surgical tools, but isnot limited thereto. For example, a variety of surgical tools may beused according to the surgical operation. Since the number of robot arms210 of the slave device 200 is limited, the number of surgical toolsmounted thereon at one time is also limited. Accordingly, when thesurgical tool needs to be changed out during surgery, the operator Sinstructs the assistant A, positioned near the patient P, to change outthe surgical tool. In accordance with the instruction, the assistant Aremoves a surgical tool 220 that is no longer needed from the robot arm210 of the slave device 200 and mounts another surgical tool 220′ placedon a tray Ton the corresponding robot arm 210.

The master device 100 and the slave device 200 may be separatelyconstructed as physically independent devices, without being limitedthereto. For example, the master device 100 and the slave device 200 mayalso be integrated with each other as a single device enclosed within asingle physical package.

As illustrated in FIGS. 1 and 2, the master device 100 may include, forexample, an input unit 110 and a display unit 120.

The input unit 110 refers to an element that receives an instruction forselection of an operation mode of the surgical robot or an instructionfor remote control of operations of the robot arm 210, the surgicaltools 220, and the image capture unit 230 of the slave device 200 whichare input by the operator S. In the present embodiment, the input unit110 may include a haptic device, a clutch pedal, a switch, and a button,but is not limited thereto. For example, a voice recognition device maybe used. Hereinafter, a haptic device will be exemplarily described asan example of the input unit 110, but a variety of devices may also beused as the input unit 110.

FIG. 1 exemplarily illustrates that the input unit 110 includes twohandles 111 and 113, but the present embodiment is not limited thereto.For example, the input unit 110 may include one handle or three or morehandles as well.

The operator S respectively manipulates two handles 111 and 113 usingboth hands as illustrated in FIG. 1 to control operation of the robotarm 210 of the slave device 200. That is, upon manipulation of the inputunit 110 by the operator S, a controller 130 may generate a controlsignal corresponding to status information of the manipulated input unit110 using a control signal generator 131 and may transmit the generatedcontrol signal to the slave device 200 through a communication unit 140.

The display unit 120 of the master device 100 may display a real imageof the inside of the patient P's body acquired by the image capture unit230, a 3D image generated using a medical image of the patient P beforesurgery, and the like. To this end, the master device 100 may include animage processor 133 to process image data received from the slave device200 and to output processed image information to the display unit 120.In this regard, “image data” may include a real image acquired by theimage capture unit 230, a 3D image generated using a medical image ofthe patient P before surgery, or a combination thereof, and the like, asdescribed above, but is not limited thereto.

The display unit 120 may include at least one monitor, and each monitormay be implemented to individually display information required forsurgery. For example, when the display unit 120 includes three monitors,one of the monitors may display the real image acquired by the imagecapture unit 230, a 3D image generated using a medical image of thepatient P before surgery, and the like, and the other two monitors mayrespectively display information regarding operation status of the slavedevice 200 or information about the patient P. In this regard, thenumber of monitors may vary according to the type, kind, or amount ofinformation to be displayed.

Here, “information of the patient” may refer to information indicatingvital signs of the patient, for example, bio-information such as bodytemperature, pulse, respiration, and blood pressure. In order to providesuch bio-information to the master device 100, the slave device 200,which will be described later, may further include a bio-informationmeasurement unit including a body temperature-measuring module, apulse-measuring module, a respiration-measuring module, an oxygensaturation-measuring module, a blood pressure-measuring module, and thelike. To this end, the master device 100 may further include a signalprocessor (not shown) to receive bio-information from the slave device200, process the bio-information, and output the resultant informationto the display unit 120.

As illustrated in FIG. 1, the slave device 200 may include a pluralityof robot arms 210, various surgical tools 230 mounted on ends of therobot arms 210, and the image capture unit 230.

The robot arms 210 may be coupled to a body 201 in a fixed state andsupported thereby as illustrated in FIG. 1. In this regard, the quantityof surgical tools 230 and robot arms 210 used at once may vary accordingto various factors, such as diagnostic methods, surgical methods, andspatial limitations of an operating room.

In addition, each of the robot arms 210 may include a plurality of links211 and a plurality of joints 213. Each of the joints 213 may connectlinks 211 and may have 1 degree of freedom (DOF) or greater. Here,“degree of freedom (DOF)” refers to a DOF with regard to kinematics orinverse kinematics. A DOF of a mechanism indicates the number ofindependent motions of a mechanism or the number of variables thatdetermine independent motions at relative positions between links. Forexample, an object in a 3D space defined by X, Y, and Z-axes has atleast one DOF selected from the group made up of 3 DOFs to determine aspatial position of the object (a position on each axis), 3 DOFs todetermine a spatial orientation of the object (a position on each axis),and 3 DOFs to determine a spatial orientation of the object (a rotationangle relative to each axis). More specifically, it will be appreciatedthat when an object is movable along each of X-, Y-, and Z-axes and isrotatable about each of X-, Y-, and Z-axes, that the object has 6 DOF.

In addition, a detector (not shown) that may detect informationindicating the state of the joint 213 may be mounted on the joint 213.For example, the detector may include a force/torque detector thatdetects force/torque information applied to the joint 213, a positiondetector that detects position information of the joint 213, and a speeddetector that detects speed information of the joint 213. In anembodiment, the speed detector may be dispensed with according to thetype of position sensor used as the position detector. FIG. 2illustrates the position sensor used as the position detector only.However, the force/torque detector and the speed detector may also beapplied thereto.

The position sensor 215 may be a potentiometer, an encoder, or the like,but is not limited thereto. The position sensor 215 may be mounted oneach joint 213 of the robot arm 210, may detect information regardingthe status of motion of each joint 213 of the robot arm 210 as describedabove, and may transmit the information to a controller 250. Thecontroller 250 that receives the detected information regarding thestatus of motion of each joint 213 of the robot arm 210 from theposition sensor 215 may calculate current position and direction of thesurgical tools 220 mounted at one end of the robot arm 210 using aposition calculator 255. In this regard, the position calculator 255applies the input information to kinematics of the robot arm 210 tocalculate the current position and direction of the surgical tools 220.However, this is an exemplary embodiment, and the calculation method ofthe position calculator 255 is not limited thereto.

As illustrated in FIG. 2, the slave device 200 may further include afirst drive unit 241 to control motion of the robot arm 210 according tothe control signal received from the master device 100.

For example, when the operator S manipulates the input unit 110 of themaster device 100, the controller 130 of the master device 100 generatesa control signal corresponding to the status information of themanipulated input unit 110 and transmits the control signal to the slavedevice 200, and the controller 250 of the slave device 200 drives thefirst drive unit 241 in accordance with the control signal received fromthe master device 100 to control motion of each joint of the robot arm210 so as to operate the robot arm 210. Here, a substantial controlprocess of, for example, rotating and moving the robot arm 210 inaccordance with the manipulation of the input unit 110 by the operator Sneed not fall within the scope of the present disclosure and thus adetailed description thereof will not be given.

Meanwhile, each joint of the robot arm 210 of the slave device 200 maymove according to the control signal received from the master device 100as described above. However, the joint may also move by external force.That is, the assistant A positioned near the operating table maymanually move each of the joints of the robot arm 210 to controllocation, orientation, and the like of the robot arm 210.

Although not illustrated in detail in FIG. 1, each of the surgical tools220 may include a housing mounted on one end of the robot arm 210, ashaft extending from the housing by a predetermined length, and an endeffector mounted on one end of the shaft.

In general, the surgical tools 220 may be classified into main surgicaltools and auxiliary surgical tools. Here, “main surgical tools” mayrefer to surgical tools including end effectors which perform directsurgical motion, such as cutting, suturing, cauterizing, and rinsing, onthe surgical region. Examples of the end effector may include a scalpel,surgical needle, clamps, a grasper, scissors, a stapler, a needleholder, a scalpel, a cutting blade, and the like, but is not limitedthereto. Any instruments required for surgery may also be used.

“Auxiliary surgical tools” may refer to surgical tools including endeffectors, such as a skin holder, which do not perform direct motions onthe surgical region but rather assist motion of the main surgical tools.

In addition, a drive wheel may be coupled to the housing. The endeffector may be driven by connecting the drive wheel with the endeffector via wire or the like and rotating the drive wheel. To this end,a second drive unit 243 to rotate the drive wheel may be mounted on oneend of the robot arm 210. For example, upon manipulation of the inputunit 110 of the master device 100 by the operator S, the master device100 generates a control signal corresponding to status information ofthe manipulated input unit 110 and transmits the control signal to theslave device 200, and the controller 250 of the slave device 200 drivesthe second drive unit 243 in accordance with the control signal receivedfrom the master device 100, so as to control the end effector in adesired manner. However, the operating mechanism of the end effector isnot necessarily constructed as described above, and various otherelectrical/mechanical mechanisms to realize required motions of the endeffector for robot surgery may also be applied thereto.

In addition, the surgical tool 220 of the slave device 200 according tothe present embodiment may be coated with a UV reactive material thatemits light in response to UV light. Here, “UV reactive material” refersto a material that is invisible under visible light but emits visiblelight when exposed to UV light.

Generally, light is not produced inside the human body. In order toefficiently perform surgical operation, light is radiated into thepatient P's body. Here, light radiated into the patient P's body isvisible light having a wavelength range visible to human eyes. Asdescribed above, when visible light is radiated into the patient P'sbody, all of organs and tissues of the patient P and the surgical tools220 inserted into the patient P's body become visible to human eyes.

Meanwhile, when UV light is radiated into the patient P's body, theinside of the patient P's body is not visible since UV light has awavelength range invisible to the human eye. However, according to theillustrated embodiment, since the UV reactive material coated on thesurgical tool 220 emits light having a visible wavelength range inresponse to the radiated UV light, the surgical tool 220 is visible tothe human eyes. That is, when UV light is radiated into the inside ofthe patient P's body, organs and tissues around the surgical region ofthe patient P are not visible, but the surgical tool 220 is visible dueto emission of light by the UV reactive material.

Accordingly, when an image of the inside of the patient P's body isacquired through the camera 231, which will be described later, an imagein which only the surgical tool 220 is visible without displaying organsand tissues around the surgical region of the patient P may be obtained.

In general, a current position and direction of the surgical tool 220that is inserted into the patient P's body to perform the surgery may beestimated by matching information calculated through the following twomethods. First, the current position and direction of the surgical tool220 may be calculated by attaching the position sensor 215 to the jointof the robot arm 210 provided with the surgical tool 220 and applyingthe status information of the joint of the robot arm 210 detectedthrough the position sensor 215 to kinematics. Second, the surgical tool220 is separated from the background image acquired by the camera, andthe position and direction of the separated surgical tool 220 arecalculated.

In this regard, in order to efficiently separate the surgical tool 220from the background image according to the second method, a marker isattached to the surgical tool 220. The marker may affect an appearanceof the surgical tool 220 and may not be recognized since the marker maybe attached to a portion of the surgical tool 220 that is hidden byimpurities such as blood, bodily fluids, and soft tissues or anothersurgical tool 220.

Accordingly, according to the illustrated embodiment, UV reactivematerial that does not affect the appearance of the surgical tool 220and that may be applied to the overall surface of the surgical tool 220is used. According to the present embodiment, the UV reactive materialmay be applied to the overall surface or a portion of the surgical tool220.

Here, applying the UV reactive material to the overall surface of thesurgical tool 220 may refer to applying the UV reactive material to theoverall surfaces of the shaft and the end effector of the surgical tool220, without being limited thereto, or applying the UV reactive materialto the overall surface of the end effector. In addition, applying the UVreactive material to a portion of the surgical tool 220 may refer toapplying the UV reactive material to a portion of the shaft, a portionof the end effector, or portions of the shaft and the end effector ofthe surgical tool 220. Here, the UV reactive material may be coated tohave a predetermined shape, without being limited thereto. In addition,when the UV reactive material is coated on a portion of the surgicaltool 220, a plurality of coated regions may be formed, but the presentembodiment is not limited thereto.

Furthermore, according to the illustrated embodiment, the slave device200 of the surgical robot may include an image capture unit 230 thatincludes a first lighting unit 233 radiating visible light, a secondlighting unit 235 radiating UV light, and a camera 231 capturing animage as illustrated in FIG. 2. Here, the first lighting unit 233, thesecond lighting unit 235, and the camera 231 may be integrated with eachother, or may be separated as physically independent devices. Inaddition, a conventional surgical robot including an endoscope and alight source radiating UV light applied to the endoscope may be used.Alternatively, a conventional surgical robot including a light sourceseparately installed from an endoscope may be used.

The first lighting unit 233 radiating visible light may be a whitelighting unit, for example, a halogen lamp, a xenon lamp, or a lightemitting diode (LED), but is not limited thereto. The second lightingunit 235 radiating UV light may be a UV lighting unit, for example, a UVlight emitting diode (UV LED), but is not limited thereto.

The camera 231 may be a stereo camera including a pair of cameras toacquire a 3D image, i.e., a left-eye camera and a right-eye camera, butis not limited thereto. According to the illustrated embodiment, thecamera 231 may also be a camera that captures a visible-light image. Inthis regard, “visible-light image” refers to an image visible to humaneyes in a visible wavelength range and may include all images acquiredunder visible light and under UV light according to the presentembodiment. Hereinafter, “image acquired under visible light” isreferred to as “first image”, and “image acquired under UV light” isreferred to as “second image”.

The camera 231 may also be a complementary metal-oxide Semiconductor(CMOS) camera and a charge coupled device (CCD) camera, but is notlimited thereto.

As illustrated in FIG. 2, the slave device 200 according to theillustrated embodiment may include a third drive unit 245, a fourthdrive unit 247, and a fifth drive unit 249 to respectively drive thecamera 231, the first lighting unit 233, and the second lighting unit235. For example, the controller 250 of the slave device 200 mayrespectively drive the third drive unit 245, the fourth drive unit 247,and the fifth drive unit 249 in accordance with the control signal fromthe master device 100 so as to operate the camera 231, the firstlighting unit 233, and the second lighting unit 235.

In this regard, the controller 130 of the master device 100 may controlon/off operations of the first lighting unit 233 and the second lightingunit 235 such that the first lighting unit 233 and the second lightingunit 235 alternately emit light. That is, the first lighting unit 233and the second lighting unit 235 may alternately emit light duringsuccessive time periods. For example, the first lighting unit 233 mayemit light during a first time period, the second lighting unit 235 mayemit light during a second time period, the first lighting unit 233 mayemit light during a third time period and so on. In an embodiment, atime period during which the first lighting unit 233 emits light may belonger than a time period during which the second lighting unit 235emits light.

Particularly, on/off operations of the first lighting unit 233 and thesecond lighting unit 235 may be controlled such that a time periodduring which visible light is radiated is longer than a time periodduring which UV light is radiated. Accordingly, the operator S andassistant A performing surgery may not recognize the radiation of UVlight. That is, the operator S and the assistant A performing a surgicaloperation need to thoroughly check the status of the inside of thepatient P's body, and such checking may be possible only while visiblelight is radiated. While UV light is radiated, only the UV reactivematerial coated on the surgical tool 220 and emitting light is visible,but organs and tissues around the surgical region of the patient P arenot visible. Thus, as a UV radiation time increases, the surgical tool220 emitting light is only visible to the operator S and the assistantA, but the inside of the patient P's body cannot be observed due to lackof radiated visible light, thereby affecting the surgical operation.Thus, the controller 130 of the master device 100 may control the timeperiod during which the second lighting unit 235 emits light to be 0.1sec or less such that the operator S and the assistant A cannotrecognize UV radiation.

Meanwhile, the controller 130 of the master device 100 may also controlon/off operations of the first lighting unit 233 and the second lightingunit 235 such that the second lighting unit 235 continuously emitslight, and the first lighting unit 233 periodically emits light atpredetermined intervals instead of alternating radiation by the firstlighting unit 233 and the second lighting unit 235 as described above.

In particular, visible light and UV light may be simultaneously radiatedinto the patient P's body, and then the first lighting unit 233radiating visible light may be turned off for a short time period andthen turned back on. In this regard, a radiation-off time of the firstlighting unit 233 may be shorter than a radiation-on time of the firstlighting unit 233. As described above, in order to prevent the operatorS and the assistant A from being aware of UV radiation, theradiation-off time may be, for example, 0.1 sec or less.

When the first lighting unit 233 and the second lighting unit 235alternately radiate visible light and UV light as described above, orwhen the second lighting unit 235 consistently radiates UV light and thefirst lighting unit 233 periodically radiates visible light atpredetermined intervals, the operator S and the assistant A do notrecognize UV radiation. However, the camera 231 may acquire images whileonly the first lighting unit 233 operates, while only the secondlighting unit 235 operates, and while both the first lighting unit 233and the second lighting unit 235 simultaneously operate. In this regard,an image acquired while visible light and UV light are simultaneouslyradiated respectively by the first lighting unit 233 and the secondlighting unit 235 may be identical to an image acquired while radiationis performed only by the first lighting unit 233.

For example, an organ 303, peripheral tissues 301, and the surgical tool220 inserted into the patient may be shown in images acquired while onlythe first lighting unit 233 operates and while the first lighting unit233 and the second lighting unit 235 simultaneously operate asillustrated in FIG. 3. Meanwhile, only a portion of the surgical tool220 coated with the UV reactive material that emits light in response toUV radiation may be shown in an image acquired while only the secondlighting unit 235 operates as illustrated in FIG. 4.

FIG. 4 exemplarily illustrates the appearance of an imagein which the UVreactive material is coated over the entire surface of the end effector221 of the surgical tool 220. However, the UV reactive material mayalternatively be coated on a portion of the end effector 221.Alternatively, the UV reactive material may be coated on portions or theentire surfaces of the end effector 221 and the shaft 223. In this case,an image different from FIG. 4 may be acquired. That is, an image inwhich only UV reactive material-coated portions are shown may beacquired during UV radiation.

As described above, when the UV reactive material is coated only on aportion of the end effector 221 of the surgical tool 220, only theportion of the end effector 221 coated with the UV reactive material isshown in an image acquired during UV radiation. Thus, the end effector221 may be efficiently separated from the background image.

That is, according to the illustrated embodiment, the surgical tool 220is separated from the background image in each of a first image and asecond image acquired through the camera 231 to estimate the currentposition and direction of the surgical tools 220 inserted into thepatient P's body. In consideration of a shape and size of the surgicaltool 220 separated from the background image and direction informationof the camera 231, the position and direction of the surgical tool 220may be calculated. In this regard, separation of the surgical tool 220from the background image may be performed by using a surgical tool 220having a specific shape, but the present embodiment is not limitedthereto.

However, since the first image includes the organ 303 and the peripheraltissues 301 as well as the surgical tool 220 as illustrated in FIG. 3,an error may occur when the surgical tool 220 is separated from thebackground image including the organ 303 and the peripheral tissues 301.Accordingly, the second image showing only the surgical tool 220 isacquired as illustrated in FIG. 4 by radiating UV light to the surgicaltool 220 coated with the UV reactive material. By using the acquiredsecond image, the surgical tool 220 may be accurately separated from thebackground image.

Meanwhile, according to the illustrated embodiment, the controller 130of the master device 100 may calculate the current position anddirection of the surgical tool 220 using only the second image (FIG. 4).For example, the controller 130 of the master device 100 may alsocalculate the current position and direction of the surgical tool 220using the following method.

That is, the aforementioned error occurring during estimation using thefirst image, in which the surgical tool 220 cannot be accuratelyseparated from the background image, may be corrected by periodicallyusing the second image while calculating the position and direction ofthe surgical tool 220 based on the first image, selected from two imagesacquired by the camera 231, i.e., the first image (FIG. 3) and thesecond image (FIG. 4). In other words, the position and direction of thesurgical tool 220 are calculated based on the image acquired undervisible light, while correcting the error occurring due to inaccurateseparation of the surgical tool 220 from the background image byperiodically using the image acquired under UV light.

In addition, the controller 130 of the master device 100 may control theimage processor 133 to process the first image acquired by the camera231, i.e., the image acquired under visible light, to display theprocessed image on the display unit 120. In this regard, when only UVlight is radiated, such as under the condition that only the secondlighting unit 235 is turned on, the controller 130 may control thedisplay unit 120 to display the first image acquired immediately beforethe second lighting unit 235 is turned on. This condition may becontinued until the first lighting unit 233 is turned back on, withoutbeing limited thereto. Accordingly, a user, for example, the operator Sand the assistant A, may continue to observe the inside of the patientP's body even when only UV radiation is being radiated.

FIG. 5 is a flowchart illustrating a method of controlling a surgicalrobot. In an embodiment, the method of controlling the surgical robotmay occur sequentially as illustrated in FIG. 5, although other ordersof operations may occur.

First, referring to FIGS. 1, 2, and 5, the surgical tool 220 coated withUV reactive material is inserted into the patient P's body to perform asurgical operation (S510). Here, “UV reactive material” may refer to amaterial that is invisible under visible light but emits visible lightwhen exposed to UV light.

In the illustrated embodiment, the UV reactive material may be coated onthe entire surface or on a portion of the surgical tool 220. In thisregard, coating of the UV reactive material on the entire surface of thesurgical tool 220 may refer to coating of the UV reactive material onthe entire surface of the shaft and the end effector of the surgicaltool 220, without being limited thereto, and may alternatively refer tocoating of the UV reactive material on the entire surface of the endeffector. In addition, coating of the UV reactive material on a portionof the surgical tool 220 may refer to coating of the UV reactivematerial on a portion of the shaft, a portion of the end effector, orportions of both the shaft and the end effector. Here, the UV reactivematerial may be coated to have a predetermined shape, but is not limitedthereto. In addition, when the UV reactive material is coated on aportion of the surgical tool 220, a plurality of regions may be coated,but the present embodiment is not limited thereto.

Then, visible light and UV light are radiated into the patient P's body(S520).

In this case, visible light and UV light may be alternately radiated,without being limited thereto. In addition, a visible light radiationtime may be longer than a UV radiation time, without being limitedthereto.

Accordingly, the operator S and the assistant A performing the surgicaloperation may be unaware of the UV radiation. That is, the operator Sand the assistant A performing the surgical operation need to thoroughlycheck the status of the inside of the patient P's body, and suchchecking may be possible only while visible light is radiated. While UVlight is radiated, only the light emitting UV reactive material coatedon the surgical tool 220 is visible, but organs and tissues around thesurgical region of the patient P are not visible. Thus, as the UVradiation time increases, the surgical tool 220 emitting light isvisible to the operator S and the assistant A, but not the organs andtissues around the surgical region and thus the status of the inside ofthe patient P's body cannot be checked. Accordingly, a relatively longduration of UV radiation may affect the surgical operation. Thus, the UVradiation time may be controlled to be 0.1 sec or less such that theoperator S and the assistant A may be unaware of UV radiation, that is,so that the UV radiation period does not detrimentally affect theability of the operator S and the assistant A to see the organs andtissues around the surgical region.

Meanwhile, as described above, visible light and UV light may not bealternately radiated. Instead, visible light may be periodicallyradiated at predetermined intervals while UV light is continuouslyradiated.

Particularly, visible light and UV light may be simultaneously radiatedinto the patient P's body, and then the first lighting unit 233radiating visible light may be turned off for a short time period andthen turned on. In this regard, a radiation-off time of the firstlighting unit 233 may be shorter than a radiation-on time of the firstlighting unit 233. As described above, in order to prevent the operatorS and the assistant A from being aware of UV radiation, theradiation-off time may be, for example, 0.1 sec or less.

Then, a first image is acquired while visible light is radiated, and asecond image is acquired while UV light is radiated (S530). That is thefirst image is acquired while visible light is radiated and UV light isnot radiated, and a second image is acquired while UV light is radiatedand visible light is not radiated.

In this regard, the first image and the second image may be acquiredusing the camera 231 of the slave device 200. In this regard, the camera231 may be a stereo camera including a pair of cameras to acquire a 3Dimage, i.e., a left-eye camera and a right-eye camera, but is notlimited thereto. According to the illustrated embodiment, the camera 231may also be a camera that captures a visible-light image. In thisregard, “visible-light image” refers to an image visible to human eyesin a visible wavelength range and may include all images acquired undervisible light and under UV light according to the present embodiment.

The camera 231 may be a complementary metal-oxide semiconductor (CMOS)camera or a charge coupled device (CCD) camera, but is not limitedthereto.

The first image and second image acquired according to this operationare as illustrated in FIGS. 3 and 4. That is, an organ 303, peripheraltissues 301, and the surgical tool 220 inserted into the patient P'sbody may be shown in the first image acquired while visible light isradiated into the patient P's body as illustrated in FIG. 3. Only thesurgical tool 220 coated with the UV reactive material that emits lightin response to UV radiation may be shown in the second image acquiredwhile UV light is radiated as illustrated in FIG. 4

As described above, the second image only displaying the surgical tool220 is acquired by coating the surgical tool 220 with the UV reactivematerial and radiating UV light into the patient P's body in order toefficiently and accurately separate the surgical tool 220 from thebackground image during a subsequent operation.

Then, a position and direction of the surgical tool are calculated usingthe acquired first image and the second image (S540). Operation S54 mayinclude the following operations.

That is, as illustrated in FIG. 6, after the first image and the secondimage are acquired (S530), the first image (FIG. 3) acquired undervisible light may be processed and displayed on the display unit 120(S541). Then, the surgical tool 220 may be separated from the backgroundimage in the first image (S542), and the first image in which thesurgical tool 220 is separated may be compared with the second image(FIG. 4) in which only the surgical tool 220 is displayed (S543).

Then, it may be determined whether the shape and position of thesurgical tool 220 of the first image are identical to those of thesurgical tool 220 of the second image (S544). When the shape andposition are the same, the position and direction of the separatedsurgical tool 220 of the first mage may be calculated (S545). When theshape and position are not the same, the shape and position of theseparated surgical tool 220 of the first image may be corrected to beidentical to those of the surgical tool 220 of the second image (S546),and the position and direction of the corrected surgical tool 220 may becalculated (S547).

That is, although the position and direction of the surgical tool 220are calculated based on the first image, an error that may occur duringseparation of the surgical tool 220 from the background of the firstimage is corrected by using the second image displaying only thesurgical tool 220.

Meanwhile, operation S540 may be performed as illustrated in FIG. 7.Particularly, after the first image and the second image are acquired(S530), the first image (FIG. 3) acquired under visible light may beprocessed and displayed on the display unit 120 of the master device 100(S541). Then, the second image (FIG. 4) in which only the surgical tool220 acquired under UV light is displayed may be processed to separatethe surgical tool 220 from the background image, and the position anddirection of the separated surgical tool 220 may be calculated (S549).That is, the position and direction of the surgical tool 220 may becalculated using only the second image that displays only the surgicaltool 220.

The aforementioned two methods are exemplary embodiments, and the methodof calculating the position and direction of the surgical tool 220 usingthe first image and the second image is not limited thereto. Inaddition, separations of the surgical tool 220 from the background ofthe first image and the second image may be performed using apre-defined shape of the surgical tool 220, without being limitedthereto.

Then, referring to FIG. 8, the position and direction of the surgicaltool 220 are calculated using the first image and the second image(S540). Information regarding the position and direction of the surgicaltool 220 is received from the slave device 200 (S550). The final currentposition and direction of the surgical tool 220 may be estimated bycombining information regarding the position and direction of thesurgical tool 220 received from the slave device 200 and the positionand direction of the surgical tool 220 calculated using the first imageand the second image (S560).

In this regard, information regarding the position and direction of thesurgical tool 220 received from the slave device 200 may be calculatedby applying the status of motion of each joint detected by the positionsensor 215 mounted on each joint of the robot arm 210 provided with thesurgical tool 220 to kinematics by the position calculator 255 of theslave device 200. However, the present embodiment is not limitedthereto.

As described above, the current position and direction of the surgicaltool 220 may be more accurately estimated by combining the position anddirection of the surgical tool 220 calculated based on kinematics bydetecting the status of motion of joints of the robot arm 210 providedwith the surgical tool 220 and the position and direction of thesurgical tool 220 calculated using the first image and the second imagein comparison with using only one method.

The methods according to the above-described example embodiments may berecorded in non-transitory computer-readable media including programinstructions to implement various operations embodied by a computer. Themedia may also include, alone or in combination with the programinstructions, data files, data structures, and the like. The programinstructions recorded on the media may be those specially designed andconstructed for the purposes of the example embodiments, or they may beof the kind well-known and available to those having skill in thecomputer software arts. The media may also include, alone or incombination with the program instructions, data files, data structures,and the like. Examples of non-transitory computer-readable media includemagnetic media such as hard disks, floppy disks, and magnetic tape;optical media such as CD ROM discs and DVDs; magneto-optical media suchas optical discs; and hardware devices that are specially configured tostore and perform program instructions, such as read-only memory (ROM),random access memory (RAM), flash memory, and the like.

Examples of program instructions include both machine code, such asproduced by a compiler, and files containing higher level code that maybe executed by the computer using an interpreter. The described hardwaredevices may be configured to act as one or more software modules inorder to perform the operations of the above-described embodiments, orvice versa. Any one or more of the software modules described herein maybe executed by a dedicated processor unique to that unit or by aprocessor common to one or more of the modules. The described methodsmay be executed on a general purpose computer or processor or may beexecuted on a particular machine such as the method(s) of controlling asurgical robot described herein.

Although a few embodiments of the present disclosure have been shown anddescribed, it would be appreciated by those skilled in the art thatchanges may be made in these embodiments without departing from theprinciples and spirit of the disclosure, the scope of which is definedin the claims and their equivalents.

What is claimed is:
 1. A surgical robot comprising a slave deviceadapted to perform a surgical operation upon a patient and a masterdevice adapted to control the surgical operation of the slave device,wherein the slave device comprises: an image capture unit comprising afirst lighting unit to radiate visible light, a second lighting unit toradiate UV light, and a camera to capture a visible-light image; and asurgical tool coated with a UV reactive material adapted to emit lightin response to UV light radiated by the second lighting unit.
 2. Thesurgical robot according to claim 1, wherein the master device comprisesa controller to control on/off operations of the first lighting unit andthe second lighting unit.
 3. The surgical robot according to claim 2,wherein: the controller controls the on/off operations of each of thefirst lighting unit and the second lighting unit to alternately radiatelight; and a radiation time of the first lighting unit is longer than aradiation time of the second lighting unit.
 4. The surgical robotaccording to claim 2, wherein: the controller controls the on/offoperation of the second lighting unit to continuously emit light and theon/off operation of the first lighting unit to periodically emit lightat predetermined intervals; and the first lighting unit has aradiation-on time longer than a radiation-off time.
 5. The surgicalrobot according to claim 2, wherein: the visible-light image comprises afirst image acquired during visible light radiation and a second imageacquired during UV radiation; and the controller receives the firstimage and second image from the camera and calculates a position anddirection of the surgical tool using the received first image and secondimage.
 6. The surgical robot according to claim 5, wherein the slavedevice further comprises: a position sensor mounted on each joint of arobot arm provided with the surgical tool to detect status informationof the joint; and a position calculator calculating the position anddirection of the surgical tool using the status information of the jointdetected by the position sensor, wherein the controller receivesinformation regarding the calculated position and direction of thesurgical tool from the slave device and estimates a current position anddirection of the surgical tool by combining the received information andthe position and direction of the surgical tool calculated using thefirst image and the second image.
 7. The surgical robot according toclaim 5, wherein the controller receives the first image and the secondimage from the camera, separates the surgical tool from a background ofthe received first image through image processing, determines whether ashape and the position of the surgical tool of the first image in whichthe surgical tool is separated are identical to a shape and position ofthe surgical tool of the second image, and calculates the position anddirection of the separated surgical tool of the first image when theshape and position are the same, or corrects the shape and position ofthe surgical tool of the first image to be identical to the shape andposition of the surgical tool of the second image and calculates aposition and direction of the surgical tool of the corrected first imagewhen the shape and position are not the same.
 8. The surgical robotaccording to claim 5, wherein: the master device comprises a displayunit to display the first image; and the controller processes the firstimage and outputs the processed image to the display unit.
 9. Thesurgical robot according to claim 8, wherein the controller outputs thefirst image acquired immediately before the second lighting unit isturned on to the display unit while only the second lighting unit isturned on.
 10. The surgical robot according to claim 1, wherein: the UVreactive material is coated on a portion of the surgical tool; and thecoated UV reactive material has a predetermined shape.
 11. The surgicalrobot according to claim 1, wherein: the UV reactive material is coatedon a plurality of portions of the surgical tool.
 12. The surgical robotaccording to claim 1, wherein the UV reactive material is coated on anentire surface of the surgical tool.
 13. A method of controlling asurgical robot comprising a slave device performing a surgical operationupon a patient and a master device controlling the surgical operation ofthe slave device, the method comprising: performing the surgicaloperation by inserting a surgical tool coated with a UV reactivematerial into a patient's body; radiating visible light and UV lightinto the patient's body; acquiring a first image during visible lightradiation and a second image during UV radiation; and calculating aposition and a direction of the surgical tool using the acquired firstimage and second image.
 14. The method according to claim 13, whereinthe radiating of the visible light and the UV light is performed byalternately radiating visible light and UV light, and a visible lightradiation time is longer than a UV radiation time.
 15. The methodaccording to claim 13, wherein the radiating of the visible light andthe UV light is performed by continuously radiating UV light andperiodically radiating visible light at predetermined intervals, and aradiation-on time of the visible light is longer than a radiation-offtime of the visible light.
 16. The method according to claim 13,wherein: the first image comprises a surgical tool, an organ, andperipheral tissue, and the second image comprises the surgical toolwithout the organ and the peripheral tissue; and the calculating of theposition and the direction of the surgical tool comprises: separatingthe surgical tool from a background of the first image; determiningwhether a shape and a position of the surgical tool of the first imagein which the surgical tool is separated are identical to a shape and aposition of the surgical tool of the second image; and calculating theposition and direction of the separated surgical tool of the first imagewhen the shape and position are the same, or correcting the shape andposition of the surgical tool of the first image to be identical to theshape and position of the surgical tool of the second image andcalculating a position and direction of the surgical tool of thecorrected first image when the shape and position are not the same. 17.The method according to claim 16, wherein the separating of the surgicaltool from a background of the first image is performed using apre-defined shape of the surgical tool.
 18. The method according toclaim 13, further comprising: receiving information regarding theposition and direction of the surgical tool from the slave device; andestimating a final current position and direction of the surgical toolby combining information regarding the position and direction of thesurgical tool received from the slave device and the position anddirection of the surgical tool calculated using the first image and thesecond image, after the calculating of the position and direction of thesurgical tool using the acquired first image and second image.
 19. Themethod according to claim 18, wherein the information regarding theposition and direction of the surgical tool received from the slavedevice is calculated by the slave device by detecting the status ofmotion of a joint of a robot arm provided with the surgical tool. 20.The method according to claim 13, wherein the calculating of theposition and direction of the surgical tool using the acquired firstimage and second image comprises processing the first image anddisplaying the processed image on the display unit of the master device.